Papers with adaptation approaches
Thesis Proposal: LLMs post-training for multilingual medical tasks. Instruction-Tuning, Continual-Pretraining or Reasoning? (2026.acl-srw)
Copied to clipboard
| Challenge: | Adapting Large Language Models to the medical domain remains an active area of research . |
| Approach: | They propose to compare three common adaptation approaches to adapt large language models to the medical domain. |
| Outcome: | The proposed models are built on top of foundational LLMs and rely on different post-training methodologies for domain and task performance. |
LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning (2023.acl-short)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have been limited when fine-tuned on small datasets. |
| Approach: | They propose to add contrastive learning to prompt-based fine-tuning to improve model performance. |
| Outcome: | The proposed approach outperforms other methods on multiple text classification benchmarks. |
Grounded Compositional Outputs for Adaptive Language Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language models are a key component of natural language processing, but their size is a problem because they are typically trained with a closed output vocabulary derived from the training data. |
| Approach: | They propose a fully compositional output embedding layer for language models that is grounded in semantically related words and free-text definitions. |
| Outcome: | The proposed model outperforms state-of-the-art methods and adaptation approaches on cross-domain modeling and cross-learning tasks. |
Simple and Effective Multi-Token Completion from Masked Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Pre-trained neural masked language models are limited to predicting a single token . recent pre-tried LMs like T5 do allow predicting multi-token completions, but are more expensive to train and run. |
| Approach: | They propose two ways to adapt pre-trained masked language models to produce multi-token completions. |
| Outcome: | The proposed method surpasses current state-of-the-art models while being more parameter efficient. |
AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prompting-based conversational query reformulation has emerged as a powerful approach for conversational search, refining ambiguous user queries into standalone search queries. |
| Approach: | They propose a framework for query reformulation using an outcome-supervised reward model via test-time adaptation. |
| Outcome: | Experiments on five conversational search datasets show that AdaRewriter significantly outperforms the existing methods across most settings. |